A camera is deemed radiometrically calibrated if there is a computational model which can predict how the raw linear sensor image is mapped to the corresponding rendered image (e.g. JPEGs) and vice versa. This mapping represents the processing that takes place in the camera's image processing pipeline.
In a typical image processing pipeline, as illustrated in FIG. 1, a raw sensor image is subjected to colour correction followed by gamut mapping and tone correction to result in a rendered image. This can be modelled mathematically as:
                              P          _                =                                            f              ⁡                              (                                  Γ                  ⁡                                      (                                          M                      ⁢                                                                                          ⁢                                              ρ                        _                                                              )                                                  )                                                    ︸                              (                                  1                  ⁢                  a                                )                                              =                                                    Γ                ⁡                                  (                                      f                    ⁡                                          (                                              M                        ⁢                                                                                                  ⁢                                                  ρ                          _                                                                    )                                                        )                                                            ︸                                  (                                      1                    ⁢                    b                                    )                                                      ≈                                          LUT                ⁡                                  (                                      ρ                    _                                    )                                                            ︸                                  (                                      1                    ⁢                    c                                    )                                                                                        (        1        )            
One particular issue with radiometric calibration is in determining the computational model.
The pipeline shown in Eqn. 1 and FIG. 1 is representative of photographic camera processing. Non photographic cameras can have simpler pipelines. Indeed, in a single sensor channel camera, the mapping from raw to rendered image is often modelled with a simple transfer curve (tone curve). The simplest colour cameras have red, green and blue sensors each of which is associated with a separate transfer curve:P=f(ρ)  (2)
In (2) the function f( ) denotes a single or three per channel tone curves. If f( ) denotes 3 curves then radiometric calibration resorts to 3 separable grayscale calibrations.
In the photographic domain (and therefore being based on a pipelines such as in Eqn. 1) it is often desirable to understand how the linear raw image is mapped to the rendered counterpart and vice versa.
Note, however, that a raw image need not necessarily be the same image that is captured by the sensor. Rather, the sensor image may be processed in a number of steps to make the image raw. Steps may include dead-pixel correction and noise removal, demosaicing and flare correction. The nature of these computations is not considered here.
Raw to Rendered
Considering the question of how to perform radiometric calibration for Eqn. 1, in the pipeline form Eqn 1 b, Chakrabarti et al [35] minimize Eqns. 3a and 3b in iteration and then solve directly for gamut mapping in 3c
                                                        min                                                          M                                      ⁢                              ∑            i                                                          ⁢                                                                                    Γ                  ⁡                                      (                                          f                      ⁡                                              (                                                  M                          ⁢                                                                                                          ⁢                                                      ρ                            _                                                                          )                                                              )                                                  -                                  P                  _                                                                    2                                              (                  3          ⁢          a                )                                                                    min                                                                          f                ⁡                                  (                                                                          )                                                                    ⁢                              ∑            i                                                          ⁢                                                                                    Γ                  ⁡                                      (                                          f                      ⁡                                              (                                                  M                          ⁢                                                                                                          ⁢                                                      ρ                            _                                                                          )                                                              )                                                  -                                  P                  _                                                                    2                                              (                  3          ⁢          b                )                                                                    min                                                          Γ                                      ⁢                              ∑            i                                                          ⁢                                                                                    Γ                  ⁡                                      (                                          f                      ⁡                                              (                                                  M                          ⁢                                                                                                          ⁢                                                      ρ                            _                                                                          )                                                              )                                                  -                                  P                  _                                                                    2                                              (                  3          ⁢          c                )            
where f( ) and Γ( ) are initialized in sensible way (e.g. to the identity function). In Eqn. 3b, f( ) is constrained to be a 7th order polynomial and monotonically increasing; and, in Eqn. 3c the Γ function is modelled with the radial basis function method of [21] where, thousands of RBFs are potentially used.
A restriction of the above optimization is presented in [8] where they optimize Eqns. 3a and 3b and ignore gamut mapping. This less general model works surprisingly well on many real pairs of raw and rendered images and this is a point we will return to later in this description. In either the full or restricted optimizations [35] or [8], the coupled nature of the equations means that although the iterative minimization converges, it is possible that a global minimum is not found. An element of search is incorporated into the iterative optimization to mitigate this problem.
The method of Kim et al. [21], which solves for the pipeline in the form of Eqn. 1 a, makes additional assumptions to decouple the optimization and, in so far as these assumptions are valid, arrive at a globally optimal solution. That method begins with the assumption that images of the same scene are captured with respect to two or more exposures. Since in a linear image, double the light equals double the response we expect that f−1(Pie)=0.5f−1(P2e) where the superscript e denotes exposure: 2e means twice as much light is captured. Debevec et al [10] showed how relating corresponding pixels under known exposure differences suffices to solve for f( ). Further in [21] it was assumed that for the set of disaturated pixels DS (those with rgbs far from the RGB cube boundary) the gamut mapping step has little or no effect and can be ignored. Relative to this assumption f( ) can be solved using the Debevec method.
Given f( ), [21] solves M− again, focusing only DS directly:
                                                        min                                                          M                                      ⁢                              ∑                          i              ∈              DS                                                                      ⁢                                                                                                        f                                          -                      1                                                        ⁡                                      (                                                                  P                        _                                            i                                        )                                                  -                                  M                  ⁢                                                                          ⁢                                                            ρ                      _                                        i                                                                                      2                                              (        4        )            
Now, with M and f( ) in hand, Γ( ) is estimated using all the pixels in the image.
                                                        min                                                          Γ                                      ⁢                              ∑            i                                                          ⁢                                                                                                        f                                          -                      1                                                        ⁡                                      (                                                                  P                        _                                            i                                        )                                                  -                                  Γ                  ⁡                                      (                                          M                      ⁢                                                                                          ⁢                                                                        ρ                          _                                                i                                                              )                                                                                      2                                              (        5        )            
In [21], Γ( ) is implemented as a multi-thousand set of radial basis functions (RBFs). Whether used in [21] or [35], a large amount of training data is required to solve for Γ( ).
It is possible to ignore the underlying structure of the colour processing pipeline and solve for the best 3D subjective function—implemented as a look-up-table that maps the raw rgbs to rendered RGBs[22], see Eqn. 1c. Again, to use this method, a large corpus of training data is required. A further complication of this latter method is that the lookup-table has (and has to have for the best performance) a non-linear grid structure [22].
Finally, in [23] a method is presented for solving for f( ) by examining the edge distribution in an image. This method has the advantage that the method works for a single image (no need for multiple exposures) but the disadvantage that the method is sensitive to processing steps such as image sharpening (used extensively in mobile phone image processing).